Linear Algebra

Mathematical discipline that deals with vectors and matrices and, more generally, with vector spaces and linear transformations

Resources

• https://en.wikipedia.org/wiki/Linear_algebra
• https://www.britannica.com/science/linear-algebra
• The Matrix Cookbook (Brandt 2012)
• Stanford, Linear algebra refresher
• Stanford CS229, algebra and calculus refresher:
  • https://github.com/afshinea/stanford-cs-229-machine-learning/blob/master/en/refresher-algebra-calculus.pdf
  • http://cs229.stanford.edu/section/cs229-linalg.pdf
• http://people.duke.edu/~ccc14/sta-663/LinearAlgebraReview.html
• https://www.khanacademy.org/math/linear-algebra
• http://nbviewer.jupyter.org/github/relopezbriega/relopezbriega.github.io/blob/master/downloads/LinearAlgebraPython.ipynb

Matrix decompositions

See:

• AI/Unsupervised learning/Dimensionality reduction and low rank modeling

• AI/Math and Statistics/SVD

• AI/Data Science/Data science applications#Recommender systems

• https://en.wikipedia.org/wiki/Matrix_decomposition

• http://people.duke.edu/~ccc14/sta-663/LinearAlgebraMatrixDecompWithSolutions.html

• http://hameddaily.blogspot.be/2016/12/simple-matrix-factorization-with.html

• https://sites.google.com/site/igorcarron2/matrixfactorizations

• http://blog.ethanrosenthal.com/2017/06/20/matrix-factorization-in-pytorch/

• Alternating Least-squares

  • Fast Python Collaborative Filtering for Implicit Datasets

• Eigen decomposition:

  • https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix
  • http://setosa.io/ev/eigenvectors-and-eigenvalues/

• LU decomposition

• QR decomposition

Code

• #CODE Tensorly
  • http://tensorly.org
  • TensorLy is a Python library that aims at making tensor learning simple and accessible. It allows to easily perform tensor decomposition, tensor learning and tensor algebra. Its backend system allows to seamlessly perform computation with NumPy, PyTorch, JAX, MXNet, TensorFlow or CuPy, and run methods at scale on CPU or GPU
  • https://github.com/JeanKossaifi/tensorly-notebooks/
  • http://tensorly.org/stable/modules/api.html#module-tensorly.decomposition
• #CODE Eigen - Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms
• #CODE https://github.com/rballester/tntorch

Books

• #BOOK Linear Algebra for Data Science (2024)
• #BOOK Templates for the Solution of Algebraic Eigenvalue Problems (Bai, Demmel 2000)
• #BOOK High Dimensional Data Analysis 2020 (HDA2020)

References

• #PAPER tntorch: Tensor Network Learning with PyTorch (Usvyatsov 2022)
  • https://www.marktechpost.com/2022/06/30/eth-zurich-ai-researchers-introduce-tntorch-a-pytorch-powered-tensor-learning-python-library-that-supports-multiple-decompositions-under-a-unified-interface/